Web-Interactive Software Testing Management Method and Computer 
System Including an Integrated Test Case Authoring Tool 



CROSS REFERENCE TO RELATED APPLICATIONS 

This application claims priority to USSN 60/430,378, filed December 3, 2002, 
herein incorporated by reference in its entirety. 

BACKGROUND OF THE INVENTION 

Field of the Invention 

[0001] The invention relates generally to the field of computer science and 
specifically to the field of software development and testing. 

Background Art 

[0002] As object-oriented software development becomes more complex, so does 
the ability to efficiently test software applications. It is estimated that more than 70% 
of the budgets of software development projects are allocated to testing, debugging, 
and maintaining the software and only 30% to the costs associated with design and 
coding. One of the problems encountered by software developers and testers 
involved in complex software projects is the insufficiency of documentation that 
conveys Requirements specifications. "Requirements" are the details describing the 
externally perceived functionality and properties of the application. Requirements 
should be clear, complete, reasonably detailed, cohesive, attainable, and most 
preferably "testable." A non-testable requirement would be, for example, the notion 
that the application is 'user-friendly, 1 which is subjective rather than objectively 
testable through end-user interaction with the application. A testable requirement, for 
example, would be one in which "the user must enter their previously-assigned 
password to access the application." 
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[0003] Determining and organizing Requirements details usefully and efficiently is 
complex. Several books are available that suggest various approaches to this task 
(See Robertson, S., et al., Mastering the Requirements Process, Addison- Wesley Pub 
Co (2000), and see Weigersm K. et al, Software Requirements, Microsoft Press 
(1999)). 

[0004] The Requirements of all potential end users should be considered when 
generating a comprehensive strategy for testing the software application. Users can 
be, for example, end-users, customer acceptance testers, customer contract officers, 
customer management, future software maintenance engineers, sales-people, or 
anyone who could later "derail" the project if their expectations are not met. In some 
organizations, requirements may end up in high-level project plans, functional 
specification documents, in design documents, or in other documents at various levels 
of detail. 

[0005] Additionally, current software testing methodologies are encumbered by 
difficulties associated with learning automation tools (e.g., WinRunner available from 
the Mercury Interactive Corporation, SilkTest available from Segue Software Inc., or 
Rational Robot available from the Rational Software Corporation), the scripting 
languages used by automation tools. 

[0006] Some attempt has been made to integrate both requirements organization and 
testing. The SilkTest automation tool, for example, comes integrated with "SilkPlan 
Pro," an application for creating a test plan and then linking the automation scripts to 
the test plan. "SilkPlan Pro" can also be used to track the automation process and 
control the execution of selected groups of test cases. One or more user defined 
attributes are assigned to each "test case" and later used in test plan queries to select a 
group of tests for execution. A test case is a file or document that describes an input, 
action, or event and an expected response, to determine if a feature of an application 
is working per the Requirements. A test case preferably contains the particulars, e.g., 
test case identifier, test case name, objective, test conditions/setup, input data 
requirements, steps, and expected results. There is also a modest capability to add 
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manual test placeholders in the test plan, and then manually add pass/fail status to the 
results of a full test run. 

[0007] The automation tool WinRunner integrates a web enabled program called 
"TestDirector;" for visually creating a test project and then linking WinRunner scripts 
to the project. "TestDirector" is a database repository based application that provides 
a variety of tools to analyze and manipulate the various database tables and test 
results stored in the repository for the project. Using a visual recorder, test cases are 
added to one or more Test Sets for execution as a group. TestDirector also provides 
user with the means for authoring manual test cases (e.g., describing each test step 
and its expected results), interactively executing each manual test, and then saving the 
pass/fail status for each test step in the repository. 

[0008] Commercially available automation tools include only rudimentary 
management and authoring features limited to isolated machines containing the 
automation tool and the software being tested. Software testing, however, is a team 
effort that requires symbiotic cooperation between the various software testers who 
need not be software developers themselves. Also, software testers need not be 
limited geographically to conduct the software testing. 

[0009] As such the present invention provides a unified multi-user platform that 
manages testing requirements, assigning test cases for execution, and includes a 
graphical user interface (GUI) driven test case authoring tool that appends test steps 
using syntax recognized by the automation tool chosen to perform the test. 

SUMMARY OF THE INVENTION 

[0010] The invention is directed to a method and computer system that include a 
web-interactive interface that permit one or more users with or without knowledge of 
test scripting to manage and run test cases on local or remote host machines. The 
management interface allows users to: (a) create their own test cases using a GUI that 
appends test case steps using a syntax recognized by the automation tool chosen to 
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perform the test (b) manage and organize Requirements visually using the web 
interface, and (c) schedule automated testing by assigning one or more test cases to be 
run on one or more host machines; the host machines including a whole or partial 
copy of the software application being tested and an automation tool for executing the 
steps of the test case. 

[001 1] One aspect of the invention is a method of software testing where a user 
accesses a web interface provided by an application server, selects a test case using 
the web interface, selects a host machine on which to execute the test case; the 
application server then transmits a files associated with a selected test case from the 
application server to the selected host machine; the files includes the steps of the test 
case to be executed by the automation tool and automation tool GUI environment 
file(s) and is parsed into an automation tool recognized test script; the host machine 
then executes the steps of the case using an automation tool loaded onto said host 
machine. 

[0012] Also provided is a method for sending updated status information about an 
executed test case and the status of its included steps from the host machine to the 
application server and providing the status information by way of the web interface. 

[0013] In an exemplary embodiment of the invention the file associated with the test 
case can be any type of file including without limitation to a flat file, markup 
language encoded file, XML file, ASCII file, HTML file, or XHTML file. 

[0014] Another aspect of the invention a test case can be scheduled (e.g., date and 
time) for execution on one or more host machines. 

[001 5] Also provided is a method for authoring a test case using a web interface 
wherein steps or procedures are added to the test case or existing steps or procedures 
are modified in the test case by choosing from a plurality of preset GUI objects or by 
adding special objects ( e.g., Database, Date, File, Loop, and Manual Step objects). 
The GUI objects being present in the application to be tested. Users can include 
actions and their expected results into the test case steps by entering one or more 
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keystrokes into a procedure field associated with the step. 

[0016] The invention further provides a computer system and method for managing 
software testing requirements wherein a users can create a requirements folder using a 
web interface and wherein the requirements folder is saved in a relational database 
coupled to an application server that provides the web interface, the user can further 
use the web interface to create a test case using a web interactive authoring tool and 
saving the test case in the relational database coupled to the application server, users 
can sort a test case within a requirements folder, and a user can create a sub- 
requirements folder using the web interface and saving the sub-requirements folder in 
the requirements folder; wherein both the requirements and the sub-requirements 
folders are saved in the relational database coupled to the application server that 
provides the web interface. 

[0017] A computer system is also provided which includes an application server 
suitable to function as a web server and suitable to provide a web interface, wherein 
the web interface allows a user to assign a test cases to a host machine for execution, 
a database operatively coupled to the application server, one or more user machines in 
communication with the application server suitable to enable the one or more users to 
interact with the web interface, and one or more host machines in communication 
with the application server, the one or more host machines comprising an automation 
tool and a full or partial copy of a computer program suitable for testing. 

[0018] The computer system also includes means for transmitting a file associated 
with a test case to a host machine; the host machine has means for parsing out the 
name of an automation tool GUI environment file(s) from the transmitted file and 
requesting the transmission of the automation tool GUI environment file(s) from the 
application server. The host machine includes means for generating a test script from 
test case step information included in the transmitted file and executing the test script 
using an automation tool. The host machine also includes means for updating the 
application server with the status of a test case and the status of any step included in 
the test case. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

[0019] The foregoing and other features and advantages of the invention will be 
apparent from the following description of exemplary embodiments and as illustrated 
in the accompanying drawings. The invention includes web-interactive interfaces. 
As such, web pages can include many GUI objects whose structure and function 
would be apparent to a person having ordinary skill in the art and are not meant to 
limit the scope of the claims. 

[0020] Figure 1 is a screenshot of a login screen by which a user accesses the web- 
interactive testing management interface. 

[0021] Figure 2 is a screenshot of the Requirements tab/Requirements List sub-tab 
of the web-interactive interface which enables a user to navigate through an 
organizational tree of one or more Requirements, any included Sub-Requirements, 
and any included test cases. 

[0022] Figure 3 is a screenshot of Requirements tab/Sub-Requirements sub-tab of 
the web-interactive interface which enables a user to navigate through a second 
organizational level of Sub-Requirement(s) included in within a Requirement. 

[0023] Figure 4 is a screenshot of the Requirements tab/Requirements Info sub-tab 
of the web-interactive interface which enables a user to view or modify the general 
description of a Requirement. 

[0024] Figure 5 is a screenshot of the Requirements tab/Test Cases sub-tab of the 
web-interactive interface which enables a user to view, copy, add, or delete a test case 
or move a test case between the Requirements/Sub-Requirements organizational 
levels. 
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[0025] Figure 6 is a screenshot of the Requirements tab/Steps sub-tab in which the 
steps (e.g., actions and their expected results) included in a test case are displayed. 
The web-interactive interface enables a user to view, move, or delete a step, or add 
new steps by way of the Integrated Test Case Authoring Tool. 

[0026] Figure 7 is a screenshot of the Run Manager tab/Case Set Info sub-tab where 
web-interactive interface enables a user to view, add, or delete a case set. 

[0027] Figure 8 is a screenshot of the Run Manager tab/Execution Info sub-tab of 
the web-interactive interface which enables a user to view the status (e.g., scheduled, 
sent to host, running, failed, passed, or stopped) of one or more test cases scheduled 
to run on a particular host machine at a designated time. 

[0028] Figure 9 is a screenshot of the Run Manager tab/History sub-tab of the web- 
interactive interface which enables a user to view the histories of test case runs by 
sorting fields by test cases, status, the user submitting the run, host machines, or run 
dates and times. 

[0029] Figure 10 is a screenshot of the Run Manager tab/Host Manager sub-tab of 
the web-interactive interface which enables users to view the list of available host 
machines whereon a test case can be run. 

[0030] Figure 1 1 is a screenshot of the Run Manager tab/Test Cases sub-tab of the 
web-interactive interface which appears when a case set is selected from the left 
frame and enables a user to add test case(s) to the case set. 

[0031] Figure 12 is a screenshot of the Run Manager tab/Run Scheduler sub-tab 
where a user may schedule runs for one or more selected test cases, assign a date and 
time for the test case to run, and assign a host machine where the selected test case is 
to be run. 
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[0032] Figure 13 is a screenshot of Run Manager tab/Manual Run sub-tab in which 
a user may execute an immediate run of a selected test case on a selected host 
machine. 

[0033] Figure 14 is a screenshot of a web-interactive interface for an Integrated Test 
Case Authoring Tool which enables a user to design new test cases or modify existing 
test cases by selecting and adding steps and by manipulating the testing procedures 
within the steps from a list of GUI objects and modules. 

[0034] Figure 15 is a screenshot of the web-interactive interface for an Integrated 
Test Case Authoring Tool which enables a user to include a step in a new or existing 
test case that checks the status (e.g., the existence) of an object. 

[0035] Figure 16 is a screenshot of the web-interactive interface for an Integrated 
Test Case Authoring Tool which enables a user to include a step in a new or existing 
test case that actuates the entry of a character string into a designated field. 

[0036] Figure 17 is a screenshot of the web-interactive interface for the Integrated 
Test Case Authoring Tool which enables a user to include a step in a new or existing 
test case derived from a Special Object. 

[0037] Figure 18 is a schematic representation of an algorithm employed by the 
invention to relay the test case associated files to the host machine, use those files to 
generate test scripts recognizable by an automation tool loaded on the host machine, 
download the appropriate automation tool GUI environment file(s) for the project, run 
the test script using the automation tool, and relay the status of the run back to the 
application server. 

[0038] Figure 19 is a schematic representation of the computer system of the 
invention. 
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[0039] Figure 20 is a schema of the relational database utilized by the application 
server. 

[0040] The foregoing figures are provided for purposes of illustration and not 
limitation and in no way limit the scope of the claims. 

DETAILED DESCRIPTION OF THE INVENTION 

[0041] For generating software testing strategies it is preferred that software testers: 
(a) obtain Requirements, functional design, internal design specifications, and other 
necessary documents, (b) obtain budget and schedule requirements, (c) hire project- 
related personnel and allocate responsibilities, reporting requirements, required 
standards and processes (e.g., release processes, change processes, etc.), (d) identify 
the higher-risk aspects of the application, set priorities, and determine scope and 
limitations of tests, (e) determine test approaches and methods - unit, integration, 
functional, system, load, usability tests, (f) determine test environment requirements 
(hardware, software, communications, etc.), (g) determine automation tool 
requirements record/playback tools, coverage analyzers, test tracking, problem/bug 
tracking, etc.), (h) determine test input data requirements, (i) identify tasks, those 
responsible for tasks, and labor requirements, (j) set schedule estimates, timelines, 
milestones, (k) determine input equivalence classes, boundary value analyses, and 
error classes,- (1) prepare a test plan document and have needed reviews/approvals, 
(m) write test cases, (n) prepare test environment and automation tools, obtain needed 
user manuals/reference documents/configuration guides/installation guides, set up test 
tracking processes, set up logging and archiving processes, set up or obtain test input 
data, (o) obtain and install software releases on various host machines, (p) perform 
tests, (q) evaluate and report results, track problems/bugs and fixes, retest as needed, 
(r) maintain and update Test Plans, test cases, test environment, and automation tools. 

[0042] The invention provides a web-interactive interface where the forgoing can be 
automated and implemented on a unified platform and shared among one or more 
users who may be separated geographically. As used herein the terms "web- 
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interactive interface," "web interactive" or "web interface" are interchangeable and 
describe or include graphical user environments provided by the application 
server/web server and which are accessible, navigable, perceivable by the one or 
more users and which are suitable for user interaction therewith using typical web 
browsers known in the art (e.g., Microsoft Internet Explorer, Netscape Navigator, 
NSCA Web Browser, Lynx, Real.com Browser, etc.), the terms also include any non- 
web based client software capable of accessing the application server and providing a 
user with a GUI with which to interact with the application server and by which to 
implement the computer system and method of the invention. 

[0043] The application server can be implemented on any general purpose computer 
that preferably includes a microprocessor (e.g., Intel, Motorola, AMD, Cyrix, etc.), 
random access and read only memory, software and data storage means (e.g., hard 
disk, floppy, CD-ROM/RW/RWRewrite, DVD-ROM/RW/RWRewrite, removable 
storage media, magneto-optical, or DAT drive), any combination of one or more 
input/output peripherals, e.g., a keyboard; pointing device or touch pad capable of 
moving a pointer and clicking at objects displayed on a video output device; a 
monitor or other video output device, and a bus to interface the forgoing components. 
The application server is preferably suitable to (a) have web hosting capability, (b) 
enable remote user login, (c) provide the web-interface to a user and include 
computer program logic enabling gathering and storing data and information from a 
database used to populate the content of the web-interface (d) to have sufficient 
memory and storage to enable multiple user logins and simultaneous multi-user usage 
(e) to have sufficient memory and storage to store automation tool GUI environment 
files associated with an application to be tested and to store test plan files which 
include textual details of the testing strategy. 

[0044] Data transmissions to and from the application server can be achieved by a 
plurality of methods known in the art, for example, a plurality of network protocols, 
e.g., IPX, Net-BUI, AppleTalk, LocalTalk, and preferably, TCP/IP, HTTP, and 
HTTPS. 
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[0045] Any operating system can be used as the platform loaded onto the 
application server, for example, Microsoft Windows 98, Windows NT, Windows 
2000, Windows XP, MacOS, UNIX (SunOS, Solaris, Linux, or any other flavor of 
UNIX), or SGI-OS, the operating system is preferably enabled or can be coupled with 
web-hosting software, the web-hosting software enabled for HTTP, HTTPS, XML, 
Java, JSP, J2EE, ASP .NET Framework, ASP, ASP.NET, Visual C# .NET, Visual J# 
.NET, Visual Basic .NET, Javascript, CGI, SSL, and SQL. 

[0046] To access the web-interface provided by the application server, one or more 
users may use a similar general-purpose computer similar to the type described above 
and preferably a general-purpose computer with web-browser software capable of 
displaying a web interface; sending data to and receiving data from the web-interface 
provided by the application server. 

[0047] Host machines also may be similarly configured to the general purpose 
computers described above and preferably include the software application being 
tested, or any logical part thereof, and a copy of an automation tool used for testing 
the software application (e.g., WinRunner, SilkTest, Rational Robot). The host 
machine is preferably intranet and extranet enabled, and most preferably also internet 
and web-enabled so that it can access the application server remotely and should 
include software suitable to receive files encoded using a plurality of markup 
languages (more infra), scripts and/or computer program code for parsing any and all 
strings in the encoded files into test scripts recognized by the automation tool, similar 
program code for encoding flat files like XML, HTML, XHTML files or any other 
flat file format that can later be decoded, and provide the automation tool with the 
parsed strings so that the automation tool can generate test scripts and run them 
against the software application being tested. 

[0048] Turning now to Figure 19, a schematic shows an exemplary computer 
system of the invention. User machine 191 belongs to a user situated in Moscow, 
Russia remotely accessing a web-interface provided by an application server 190 
located in San Francisco, California. User machine 192, and 193, situated in Sydney, 
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Australia and San Francisco, California, respectively, similarly access the web- 
interface provided by application server 190. It is within the scope of the invention 
and apparent to the skilled artisan that users may access an application server 190 
remotely from any geographic location by way of the internet, by Wide Area 
Network, by Local Area Network, or by a physical cable mediated interface directly 
to application server 190. 

[0049] Application server 190 having coupled database 190A (which could be 
housed on a separate computer or on the same computer as application server 190 and 
which may be a physical component of application server 190) dispatches data to and 
receives data from various host machines, for example, Host machines 194, 195, and 
196 located in Tokyo, Japan; Bombay, India; and San Diego, California, respectively. 
It is within the scope of the invention and it would be apparent to the skilled artisan 
that host machines may access application server 190 remotely from any geographic 
location via the internet, via a Local Area Network, via a Wide Area Network, or by a 
physical cable mediated interface with the application server 190, itself. In an 
exemplary embodiment, the user machine(s) transmit data between the application 
server, while being prevented from "seeing" or accessing the host machines by a 
firewall provided by the application server. In this embodiment the host machines 
communicate solely with application server 190. Users initiate test case executions 
first by submitting requests to application server 190. Host machines detect that a test 
case has been scheduled for execution thereon by accessing the information from the 
application server 190 (more infra). 

[0050] The data transmissions between application server 190 and host machines 

194-196 preferably include the transmission of test cases, the included steps and 

procedures of which are encoded into an XML file and parsed by a script and/or other 

computer program code loaded on the host machine into strings readable by the 

software testing automation tool which generates a test script from the parsed XML 

file. Upon running the test script, the host machine transmits the status of the 

executed test case and/or its included steps back to the application server. Test status 

can include, for example, that the test executed successfully, that the test failed, that 
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the test has been stopped, or that the test has been paused by a user managing the test 
from the application server. 

[0051] Web-Interactive Software Testing Management 

[0052] Turning now to Figure 1, a user accesses the web-interactive interface by 
logging into the web-interface by, for example, entering their company name into 
field 11, their User Name into field 12, and their Password into field 13. Typical web 
browser software known in the art includes functionality that enables users to re-login 
to a website by way of application server generated cookies, histories, recalled 
keystrokes or other means by which a user's login and password is recalled 
automatically. Other software functionality (e.g., macros, login scripts, or other 
means of authentication) that enables access by automatic login can be used. 

[0053] Upon authenticating to the application server, a user is granted access to the 
web-interface depending on their respective permissions and their security level. This 
may grant a user access to certain Projects while restricting them from others. 
Additionally, this may restrict a user's access to certain Requirements, Sub- 
Requirements, and/or test cases within the Sub-Requirements. 

[0054] Turning now to Figure 2, the application server provides the user with a web 
interface for organizing a testing strategy for a particular application. Each 
application intended to be tested or managed can be selected as a "Project" using pull- 
down menu 20 and preferably consists of a one or more Requirements, one or more 
Sub-Requirements included in the Requirement(s), and one or more test cases 
included in each Sub-Requirement(s), wherein each test case comprised a series of 
sequential steps and procedures used to test the whole application or segments of 
application code. While Requirements tab 21 and Requirements List sub-tab 21 A are 
selected and remain active, a user can view Requirements Tree 22 showing the 
organization of each Requirement and its included Sub-Requirements. For example, 
as shown in Requirements Tree 22, there are three Requirements folders: "1 .Test 
Login," "2. Test Requirements," and "3. Test Topnav window." The Requirements 

folders can be the highest and most general level of organization of functionality in an 
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application sought to be tested. Requirements Tree 22 provides collapsible lists in 
which a user may expand and collapse Requirements folders and any included Sub- 
Requirements. 

[0055] Table 23 provides a listing of available Requirements for a selected Project 
and provides the user with check boxes 23A with which to select any one 
Requirement in order to manipulate it. By selecting a Requirement by way of a 
check-box, a user may delete a Requirement from the Project by subsequently 
clicking Delete Button 25, or a user may which to add a new Requirement for testing 
by clicking Add New Button 24. If a user selects to make a new Requirement they 
are prompted with a screen (not shown) with fields in which the name of the 
Requirement and a brief description can be entered. 

[0056] Depending on the organizational strategy chosen, a user may transplant one 
Requirement into another, thus making it a Sub-Requirement, by clicking on the one 
or more check-boxes provided in Table 23 corresponding to the one or more 
Requirements they which to transplant and using pull-down menu 26 to indicate the 
destination Requirement into which to transplant the selected one or more selected 
Requirements, and subsequently by pressing Go Button 27. 

[0057] Sub-Requirements can be similarly manipulated as shown by selecting the 
Sub-Requirements sub-tab 30 of Figure 3. If a user where to click on any one 
Requirement, such as depicted by highlighted Requirement "Test Requirement" 31, a 
user would be able to view the Sub-Requirements 32 contained therein. Sub- 
Requirements Table 32A provides similarly check-boxes 32B as Requirements Table 
23 from Figure 2, where a user can select from one or more Sub-Requirements by 
clicking on the provided check-boxes 32B, Add a new Sub-Requirement using Add 
New Button 33, delete a Sub-Requirement from the tree using Delete Button 34, or 
transplant one Sub-Requirement into another using pull-down menu 35 and then 
executing the transplant using Go Button 36. 

[0058] Turning now to Figure 4, a user may click a Sub-Requirement to view its 

contents such as the highlighted Sub-Requirement "Login GUI functionality" 41. By 
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clicking on the Requirements Info sub-tab 42, users may view information relating to 
the selected Sub-Requirement as shown in fields 44, and may edit the fields' contents 
using Edit Button 45. When a Sub-Requirement is highlighted, such as Sub- 
Requirement "Login GUI functionality" 41, a user may view all of the test cases 
included within the highlighted Sub-Requirement as shown in window 43. 
Furthermore, a user may click on the test case(s) themselves to view the steps and 
procedures that the automation tool would carry out as derived from the test case. 

[0059] Test cases can optionally be manipulated, as shown in Figure 5, using Test 
Case sub-tab 50. A user may click on the Sub-Requirements containing the one or 
more test cases. For example, the Sub Requirement "Incorrect Entries" 51 is 
highlighted as shown and includes two test cases that are shown in window 52. A 
user may further manipulate the test cases using Test Case Table 53, which provides 
the user with click-boxes 53A to select from the available test cases, add a new test 
case (either an automated or manual test case, more infra) using Add Button 54, 
delete a checked test case using Delete Button 56, or transplant a selected test case 
into a Requirement or Sub-Requirement folder selected from pull-down menu 57 and 
pressing the Go Button 58. Additionally, users may clone one or more test cases that 
they may which to run under other Requirements or Sub-Requirements by selected 
the desired test case(s) and clicking the Copy Button 55. The test case replica may 
then be transplanted into a Requirement or Sub-Requirement using the features 
described above. Information about test case(s) can be viewed in "Description" 
column 53B and "Status" column 53C. Test case steps, which include actions, 
procedure, and expected results, can be added or modified using the Integrated Test 
Case Authoring Tool discussed infra. 

[0060] At the most specific level of organization, user(s) may manipulate the steps 
and procedures of a test case as shown in Figure 6. User(s) may click on the test 
case(s) they wish to manipulate, for example, Test Case 61 called "Correct 
password/Incorrect Username," located in the "Incorrect entries" Sub-Requirement of 
the "Test Login" Requirement. A table 62 shows the sequential steps 62B of the Test 
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Case 61, the Action 62 C performed at each step, and the Expected Result 62D of 
Action 62C being performed by the Automation tool on the Application being tested. 

[0061] Clicking the corresponding check-box 62 A and clicking the delete button 64 
can delete one or more steps of the test case. New steps and procedures can be added 
by pressing Add New button 63 and using the Integrated Test Case Authoring Tool 
shown in Figures 15-18 {infra). The text of each step specifies an Action 62C using a 
character string recognizable by an automation tool, for example, the character strings 
"txtPassword" and "txtUser" as shown in the column Action 62C are character strings 
recognized by automation tool and indicate (GUI object) fields in the software 
application sought to be tested application) where a password and a user name can be 
entered, respectively. This text can be clicked on by a user and edited using the 
Integrated Test Case Authoring Tool. The skilled artisan will be able to modify the 
Integrated Test Case Authoring Tool to use the scripting language of any automation 
tool {e.g., WinRunner, SilkTest or Rational Robot) without undue experimentation. 
To prevent inconsistencies and to maintain test case integrity, a computer program 
code included in the application server {e.g., Active Server Page [ASP] scripting) 
regulates the permissions of test case(s) that is disallow the editing of a test case when 
it has been scheduled to run on a host machine or, conversely, to disallow scheduling 
while a test case is edited. 

[0062] In order to run a test case(s) on a host machine(s), a user may utilize the 
web-interface provided by the Run Manager tab 70 as shown in Figure 7. Window 72 
shows lists all available Case Sets. Case Sets comprise one or more test cases 
selectable from the Requirements and Sub-Requirements. Table 73 lists the Case Sets 
which can be selected using their corresponding check-boxes 74. Selected Case Sets 
can be deleted using the Delete Button 76 or a new Case Set can be created using the 
New Button 75. 

[0063] Once a Case Set is created it is preferably filled with one or more test cases 
intended for execution. Turning now to Figure 1 1, if the Case Set "Requirements" 
111 is selected, it can be filled with one or more test cases selected from 
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Requirements Tree 113. When a Requirement or Sub-Requirement is highlighted 
from Requirements Tree 113, any test case(s) included therein is displayed in the 
lower window 113A and are transferable into the selected Case Set. For example, in 
Figure 1 1, the test case "Correct User Name & Password" is included in Sub- 
Requirement "1.2.1 Correct Entries" which is a Sub-Requirement included in "1.2 
Login security function" which is in turn included in Requirement "Test Login." 
Case Sets can be filled by the selecting on transfer icon 114, which actuates the 
copying of a test case into the selected Case Set and the test case appears in Case Set 
table 112. 

[0064] Turning now to Figure 12, a user may schedule a test case for execution on a 
host machine using the Run Scheduler sub-tab 120. Choosing from the available 
Case Sets 121, a user is provided with a table 122 of the test cases included in the 
selected Case Set available for execution. By selecting one or more test cases using 
the check-boxes 123, a user may assign the test case to a host machine selected from 
pull-down menu 124 and the date and time when the test case is to run using pull- 
down menus 125 and 126, respectively. After a user has selected the desired 
parameters for test case execution they may click the Submit button 127 rendering the 
test case available to the host machine upon host machine query. 

[0065] Turning now to the Host Manager sub-tab 100 as shown in Figure 10, a Host 
Machine Table 101 is displayed that allows a user to add or remove host machines. 

[0066] Turing now to the schema shown in Figure 18, host machine 185 includes 
command manager 188 which is preferably computer readable code that reads in 
decoded/parsed files and can subsequently pipe them, if appropriate, into one of a 
suite of functions, scripts, executables, which are preferably DLL files. The functions 
can be, for example, an environment setup creator 189, a test case steps manager 
1810, and a task query 1811 (more infra). 

[0067] Communication between application server 180 and the automation tool 

included in host machine 185 can be achieved using a request/response network 

communication model, wherein, the Simple Object Access Protocol (SOAP) protocol 
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is preferred. The benefits of SOAP are (a) HTTP communication; an RPC-like 
protocol that is simple, well known in the art and more likely to function in the face 
of firewalls than other protocols known in the art. HTTP requests are typically 
handled by web server software of application server 180, such as IIS and Apache, 
and (b) platform-neutral data representation protocol like XML. There are numerous 
XML encoding and decoding software products available for every programming 
environment and platform. 

[0068] In another embodiment, Remote Procedure Call (RPC) protocols could be 
used, for example, Sun RPC, DCE RPC, Object RPC (ORPC) protocols like DCOM 
and CORBA's Internet Inter-ORB Protocol (HOP) or General Inter-ORB Protocol 
(GIOP), custom protocols like datagrams (text or binary message) exchange over 
network (HTTP, SMTP/POP, TCP/IP, named pipes and so on) also may used. 

[0069] These communication protocols and data transmission means allow 
Application Server 180 to control and manage remotely automation tool setup testing 
environments, implementation and execution of testing, gathering information about 
the software application to be tested and to return the result of these operations. 

[0070] A task query 1811 function (e.g., DLL file, executable script, callable script 
or computer program), can be called at any predetermined time interval by command 
manager 188 to query application server 180 if any test cases have been scheduled to 
be run on host machine 185. A query can be sent by generating a query file including 
the query string by host encoder 186. Preferably this query file takes the form of a 
flat file or XML file or any other format known in the art capable of transmitting a 
query instruction (e.g., HTTP or HTTPS request). It will be appreciated by the 
skilled artisan that any callable script or computer program can be written using only 
due experimentation to facilitate this query function. 

[0071] If the query returns a response from application server 180 that a test case 

has been assigned and scheduled to host machine 185, task query 1811 transmits a 

"send" request to application server 180 which actuates the transmission of one or 

more files associated with the assigned and scheduled test case. The "send" 
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instruction can also be generated by host encoder 186 and take the form of an XML 
file, flat file, or any other file capable of transmitting a send instruction. The send 
initiates a gathering of information about the assigned and scheduled test case from 
database 181 coupled to application server 180. Information gathered from database 
181 is aggregated and one or more files are generated from the gathered information 
by server encoder 184. Application server 180 can communicate with, gather data 
from and input data into coupled database 181 and further populate the web-interface 
with from database 181 using a variety of methods known in the art including without 
limitation to SQL (e.g., Sybase, Oracle, or Microsoft SQL) or any other relational 
database query language, cgi scripting, PERL scripting and, JAVA, and Javascript. 

[0072] A test case file preferably includes the steps (actions, procedures, and 
expected results) of the test case to be executed and the name of an automation tool 
GUI environment file(s) associated with the application testing project. The one or 
more test case files can take the form of an XML file, flat file, or other file (e.g., 
HTML, SGML (Standard Generalized Markup Language), XHTML (Extensible 
HyperText Markup Language) capable of transmitting information suitable for 
parsing by host decoder 187. 

[0073] Once a file associated with a test case has been received by host machine 
185, host decoder 187 reads the file and parses out information for the execution of 
the test case, for example, the name of the automation tool GUI environment file(s) 
associated with the project, the test case steps and their dependencies, and execution 
sequences used by the automation tool. The name of the automation tool GUI 
environment file(s) is read by environment setup creator 189 which then sends a 
request to the application server 180 to send and a copy of that automation tool GUI 
environment file stored by application server 180 to host machine 185. Environment 
setup creator 189, upon receipt of the automation tool GUI environment file by host 
machine 185, loads the automation tool GUI environment file into the automation 
tool. 
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[0074] The automation tool GUI environment file(s) (e.g., a "gui" file recognized 
by the WinRunner automation tool, or a ".inc" file recognized by the SilkTest 
automation tool) contain the description (e.g., type, name, size, position or location on 
the screen, and default setting) of any GUI objects (e.g., radio-boxes, check-boxes, 
buttons, pull-down menus, hyperlinks) as well as any calls and properties of the 
objects the test case will be testing. 

[0075] Both WinRunner and SilkTest automation tools use a "lookup table" 
mechanism to isolate the variable name used to reference a GUI object in a test script 
from the description used by the operating system to access that object at runtime. 
SilkTest normally places an application's GUI declarations in a test frame file (a 
".inc" file). There is generally one GUI declaration for each window and each object 
in a window. A GUI declaration consists of an object identifier — the variable used in 
a test script — and its class and object tag definition used by the operating system to 
access that object at runtime. SilkTest provides the following capabilities to define an 
object tag: (1) a string, which can include wildcards; (2) an array reference which 
resolves to a string which can include wildcards; (3) a function or method call that 
returns a string, which can include wildcards, (4) an object class and class relative 
index number; and (5) multiple tags each optionally conditioned with (6) an 
OS/GUI/browser specifier. WinRunner normally places an application's logical 
name/physical descriptor definitions in a GUI Map file (the ".gui" file). There is 
generally one logical name/physical descriptor definition for each window and each 
object in a window. The logical name is used to reference the object in a test script, 
while the physical descriptor is used by the operating system to access that object at 
runtime. WinRunner provides the following capabilities to define a physical 
descriptor: (1) a variable number of comma-delimited strings that can include 
wildcards, where each string identifies one property of the object. The notion behind 
this lookup table mechanism is to permit changes to an object tag (SilkTest), or a 
physical descriptor (WinRunner) definition without the need to change the associated 
identifier (SilkTest) or logical name (WinRunner) used in the test case. In general the 
object tag (SilkTest) or physical descriptor (WinRunner) resolve to one or more 

-20- 



Ret SYMB-001UTL 



property definitions that uniquely identify the object in the context of its enclosing 
parent window at runtime. It is also possible with both automation tools to 
dynamically construct and use object tags (SilkTest) or physical descriptors 
(WinRunner) at runtime to reference objects in test scripts. 

[0076] The automation tool GUI environment file(s) is used along with the 
automation tool readable test script generated by host decoder 187 by parsing test 
case steps from the test case associated file sent to host machine 185 by application 
server 180. Examples of shareware XML parsing scripts are those available from 
Mozilla (http://www.mozilla.org/ rdfMoc/xml.html /ns/modules/xml/expat), and the 
associated APIs at expat/xmlparse/xmlparse.h and the glue code available at 
ns/modules/xml/glue/. The automation tool then reads both the automation tool GUI 
environment file(s) and test script and executes the script. 

[0077] Now turning to Figure 8, the Execution Info sub-tab 80 under the Run 
Manager tab 70 (Figure 7) shows displays an Execution Information Table 81 
wherein a user can monitor the Status 84 of a scheduled Test Case 83. The column 
Status 84 can indicate that a test case has been scheduled (Date 87, Time 88) 
scheduled, sent to host, running, failed, passed, or stopped. Additionally, the web 
interface permits users to monitor the status of each step of each test case by clicking, 
for example, on an expandable list button (not shown) showing a list of the test case 
steps, their execution times and current status. 

[0078] The User Column 85 indicates which user submitted the test case for 
execution to which Host machine 86. This feature permits one or more users on the 
system to monitor each other's activities thus better dividing up the testing work load, 
keeping each user informed of the Project's progress, increasing testing efficiency, 
and aiding in subsequent testing strategies since all necessary information is centrally 
provided through the web-interface. As with the previously described features, one or 
more Test Cases 83 can be selected using check-boxes 82, and the execution of the 
selected Test Cases 83 can be stopped using the Stop Button 810; assuming a user has 
sufficient access to do so. The History column 89 allows the user to click on any of 
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the bolded History hyperlinks and be taken to the History sub-tab 90 view as shown 
in Figure 9. 

[0079] Optionally, a user may submit a test case for immediate execution as shown 
in Figure 13 using the Manual Run sub-tab 130 by selecting the test case to be run 
from pull-down menu 132 and selecting the host machine where the test is to be run 
from pull-down menu 131 and clicking on Run Button 133 or terminating a test case 
run with Stop Button 134. 

[0080] The History sub-tab 90 displays a page, as shown in Figure 9, that allows a 
user to search through the histories of test case execution and sort the views by Test 
Case 92, by Status 93, by the user who submitted the test case for execution 94, by 
Host 95, or by date or date range 96 or all dates 97. 

[0081] The application server 180 can update the web interface with status 
information provided by host machine 185, as shown in Figure 18, with current status 
information regarding an executed test case or step at any predetermined time 
interval. The application server 180 can achieve such a status refresh using a 
JavaScript, Java, ASP, JSP6, C#, J#, VB.NET, ASP .NET or CGI based program that 
query the host machine and obtain current information about the status of one or more 
test cases and/or their included steps submitted for execution. Alternatively, the host 
machine 185 can contain a callable function like test case step manager 1810 as part 
of command manager 188 which sends a host encoder 186 generated file containing 
status information to application server 180, which file is parsed by server decoder 
183 and the information therefrom piped to the web interface for display to one or 
more users. 

[0082] Application server 180 and a coupled database 181 (which could be housed 
on a separate computer or on the same computer as application server 180) includes 
data and information used to populate the fields of the application server and 
provided logical links to one or more files associated with a test case and test plan 
{e.g., automation tool GUI environment files, Test Plan files, Attachment files). For 



-22- 



Ref. SYMB-001UTL 



example, the database coupled to the application server can be a relational database as 
shown in Figure 20 and can include, for example, tables such as those in Table 1 : 

Table 1 : Relational Database 



Table Name 


Contents 


FUNCTION_OBJECT 


Links between automation functions and predefined objects 


PRIORITY 


List of priorities 


STATUSES 


List of stages of objects using 


APPLICATIONS 


Information relating to project(s) 


USERS 


Information relating to user(s), their login name, password, 
phone number 


USER_TO_APP 


Information relating to user role in project(s) 


ROLES 


List of users roles 


NOTES 


Notes (task, questions.) and information relating to notes 
(created date, author, theme). 


APPLICATIONFOLDERS 


Information relating to requirement(s) 


GUIMAP 


Information relating to automation tool GUI environment 
file(s) 


GUI_OBJECTTYPES 


List of predefined objects 


GUI_WINDOWS 


Information relating to window(s) in current GUIJMAP , 
name, physical description 


GUI_OBJECTS 


Information relating to object(s) in current GUI_MAP , name, 



-23- 



Ref. SYMB-001UTL 





physical description, parent window 


tmp_GUI_OBJECTS 


Information relating to object(s) in GUI MAP , name, physical ! 
description, parent window J 


TEST CASES 


Information relating to test case(s) 



[0083] Integrated Test Case Authoring Tool 

[0084] The computer system including the application server that provides user(s) 
with web-interactive interface to the user also provides a web-based GUI interface for 
use in authoring test cases. The steps and procedures encoded into the test cases 
utilize terminology, text strings, and/or nomenclature native to an automation tool so \ 
that the steps and procedures can be easily converted into an automation tool :| 
recognized test script upon parsing. 

[0085] Automation tools provide proprietary, interpreted, scripting languages that 
control constructs, arithmetic and logical operators, and a variety of built-in library 
functions to perform such activities such as string manipulation, regular expression 
support, or standard input and output, etc. 

[0086] SilkTest, for example, provides a strongly typed, object-oriented 
programming language called "4Test." Variables and constants in 4Test may be one 
of 19 built-in data types, along with a user defined record data type. 4Test supports 
single- and multi-dimensional dynamic arrays and lists, which can be initialized ! ! 

statically or dynamically. i 

[0087] WinRunner provides a non-typed, C-like procedural programming language 
called "TSL." Variables and constants are either numbers or strings. TSL supports 
sparsely populated associative single- and pseudo multi-dimension arrays, which can 
be initialized statically or dynamically — element access is always done using string 
references — foobarfT'] and foobar[l] access the same element. 
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[0088] A user with no scripting experience, relying only on the web-interactive 
interface provided by the computer system can generate test cases by clicking on GUI 
objects and entering test strings into appropriate fields that require user input. 
Furthermore, users can generate test cases using a combination of manual and GUI 
controlled step and procedure addition. 

[0089] Turning to Figure 6, user(s) may click on either the Add New Button 63, 
thus creating a new step in the selected test case, or on the bolded-hyperlinked text of 
a test case step to modify that step. Users may append steps to existing steps and 
procedures of a test case or may modify existing steps using the Integrated Test Case 
Authoring Tool interface as shown in Figure 14. This feature of the invention 
provides users with the option to choose from a plurality of preset GUI objects 141, 
each having embedded modules and procedures. Users may also choose from a 
plurality of preset "Special Objects" 142 {e.g., objects that do not map to a GUI 
objects), for example, a database object, a date object, a file object, or a loop object. 
Additionally, users may still create "manual" test cases (as opposed to "automated" 
test cases which utilize the plurality of preset GUI objects) despite the absence of an 
automation tool. Users may click on the "Manual Step" Special object icon and enter 
any text in customized action and customized expected result fields (not shown). Test 
cases created as "manual" test cases; which may include "Manual Step" Special 
object as appended using the Integrated Test Case Authoring Tool; cannot be 
executed using in an automated fashion by generating a test script and executing the 
test script using an automation tool. Manual test cases can be manually executed. 

[0090] Both WinRunner and SilkTest automation tools, for example, support the use 

of class mapping (e.g., mapping a custom class to a standard class with like 

functionality) along with a variety of X:Y pixel coordinate clicking techniques to 

handle bitmap objects, as well as the use of external DLL functions to assist in object 

identification and verification. Beyond these shared features each automation tool has 

the following unique custom capabilities: (a) SilkTest has a feature to overlay a 

logical grid of X rows by Y columns on a graphic that has evenly spaced "hot spots" 

this grid definition is then used to define logical GUI declarations for each hot spot. 
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These X:Y row/column coordinates are resolution independent (e.g., the logical 
reference says "give me 4 th column thing in the 2 nd row", where that grid expands or 
contracts depending on screen resolution), (b) WinRunner has a built-in text 
recognition engine which works with most standard fonts. This capability can often 
be used to extract visible text from custom objects, position the cursor over a custom 
object, etc. This engine can also be taught non-standard font types. 

[0091] Upon selecting a GUI object whose functionality is sought to be tested and 
including in the step the action, procedure, and expected result associated with that 
GUI object, the object information appears on the web-interface. For example, 
Figure 14 shows that the test case "Correct Username/Incorrect Password" 148 is 
selected for modification, and that the user has chosen to append a step after step 4 as 
shown in field 143. The GUI object chosen for this step is the "txtPassword" object 
(the standard syntax used by an automation tool to indicate a password text entry 
field) as indicated in field 144. Next, the user chooses from a list of preset testing 
procedures associated with the selected object provided in field 145. By clicking the 
Submit Button 146, a user adds the GUI object and any associated procedures to the 
test case, or may alternately cancel addition of the step by clicking the Cancel Button 
147. 

[0092] Procedures capable of being tested vary between automation tools and are 

known to persons having ordinary skill in the art. As in Figure 15, for example, if a 

user selects the "Clear" object 150 for inclusion as a test case step, the user may then 

choose the procedure "Check if button exists" 151 for which the expected result 152 

is a positive or negative response whether an object exists. These types of "check" 

procedures run without user interaction with the application. Alternatively, a user 

may append or modify a step to include an object and associate procedure in which 

invites user-application interface, for example, entering text into a field, as shown in 

Figure 16. Here, the GUI object "txtPassword" 160 is selected to be an included step 

in test case "Correct Username/Incorrect Password" 148 (Figure 14). A user may then 

select a procedure, for example, the "Enter a random string" 161 procedure providing 

the user with field 162 for entering one or more keystrokes (e.g., single character or 
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character string in any language, number, floating number, decimal, symbol(s), or a 
combination thereof) and field 163 for specifying an integer value for the length of 
the character string. 

[0093] The Integrated Test Case Authoring Tool further provides a user with the 
ability to choose from a list of Special Objects for inclusion as steps in a test case. 
For example, Figure 17 shows the selection of the Special Object "Database" 171 and 
its associated procedure "Execute SQL query" 172, for which a user may specify an 
SQL query 173. This and other Special Objects will be apparent to the skilled artisan 
and can be adjusted to suit the specific automation tool performing the test and 
similarly included in the web-interface for GUI base user authoring without undue 
experimentation. 

[0094] The references cited above are all incorporated by reference herein in their 
entirety, whether specifically incorporated or not. 

[0095] Exemplary embodiments of the invention have been discussed in detail 
above. While specific embodiments are discussed, it should be understood that this is 
done for illustrative purposes only. The skilled artisan will recognize that other 
components, configurations, or combinations can be used without parting from the 
spirit and scope of the invention. Thus, the breadth and scope of the present invention 
is defined in accordance with the appended claims and their equivalents. 
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